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ABSTRACT ; We  consider  ill-posed  problems  of  the  form 


where  g is  given  and  we  must  compute  f.  The  Tikhonov  regularization 
procedure  replaces  (1)  by  a ^ne-parameter  family  of  minimization  prob' 
lems  — Minimize  (| |Kf  - g| | + afi(f))  — where  is  a smoothing  norm 

chosen  by  the  user.  We  demonstrate  by  example  that  the  choice  of 
is  not  simply  a matter  of  convenience.  We  then  show  how  this  choice 
affects  the  convergence  rate,  and  the  condition  of  the  problems  gen- 
erated by  the  regularization.  An  appropriate  choice  for  Q depends 
upon  the  character  of  the  compactness  of  K and  upon  the  smoothness 
of  the  desired  solution. 


t The  material  in  this  paper  was  presented  at  the  1976  Fall  SIAM 
meeting,  Atlanta,  Ga.  October  16  - 18,  1976. 
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1.  Introduction 

We  consider  ill-posed  problems  of  the  following  type.  Given  a 

function  g(t),  0<t<l,  find  a function  f(t)  such  that 

/^K (t , s) f (s) ds  = g(t),  0<tsl.  (1) 

0 

In  (1)  we  assume  K is  square-integrable  and  its  null  space,  N(K)  = {O}. 
For  example,  if 

K(t,s)  = 1 t>s  1 , (2) 

0 t<s  ' 

then  the  solution  of  (1)  is  the  derivative  of  g. 

From  (1)  using  quadrature  rules,  we  can  generate  — in  general 
rectangular  but  typically  square  — linear  algebraic  systems 

Ax  = b,  (3) 

where  A has  full  rank  and  whose  solutions  approximate  the  solution  of 
(1)  . 


A problem  is  ill-posed  in  the  Hadamard  sense,  if  its  solution  does 
not  depend  continuously  upon  the  data.  Equations  of  type  1 are  ill- 
posed.  Note,  however,  that  the  associated  algebraic  systems  (3)  are 
well-posed,  since  A has  full  rank.  However,  for  numerical  computations 
problems  must  not  only  be  well-posed,  but  also  well-conditioned.  Small 
variations  in  the  problem  data  must  not  yield  large  variations  in  the 
solution.  If  we  obtain  the  problem  in  (3)  by  applying  standard  quad- 


j 


f. 

rature  rules  to  (1) , it  is  easy  to  demonstrate  that  we  obtain  ill-condi- 


tioned algebraic  systems.  This  happens  because  the  operator  in  (1) 
is  compact;  i.e.,  its  singular  values  + 0 as  n+  “.  Recall  that 
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each  such  operator  has  a singular  value  decomposition  Smithies  ll], 

oo 

r T 

K = ) o V u . (4) 

n n n 

n=l 

The  functions  u^  and  v^,  n=l,  2,...  in  (4)  are  eigenvector  systems 

2 

respectively,  for  the  symmetric  operators  K*K  and  KK*.  The  a are  the 

n 

eigenvalues  of  K*K  and  the  are  called  the  singular  values  of  K. 

Equality  in  (4)  is  in  the  sense. 

Since  the  matrices  A approximate  K and  o 0,  the  singular  values 

n 

of  A will  also  converge  to  zero  as  the  approximation  to  K is  refined. 

Thus,  A will  become  ill-conditioned.  We  note  that  for  crude  approxima- 
tions, A may  be  fairly  well-conditioned. 

Our  discussion  focuses  on  the  Tikhonov  regularization  procedure, 
Tikhonov  ( 2 ] . 

Definition  1.  We  say  a one-parameter  family  of  problems,  denoted  by 
P(a)  for  OSoSl,  is  a regularizing  family  for  a problem  P if 

(a)  For  O^aSl,  each  P(a)  is  well  posed;  and 

(b)  The  solution  f^  of  P(a)  converges  to  the  solution 
of  the  original  problem  as  a + 0. 

Convergence  of  the  solutions  is  measured  in  some  relevant  norm.  For 
numerical  work  we  want  at  least  pointwise  convergence. 

For  an  equation  of  type  (1),  Tikhonov  [2]  introduces  the  following 
regularizing  family 

P(a)  : Minimize  | |Kf  - g|(^  + aft(f).  (5) 

In  (5),  11*11  is  typically  an  L^-norm,  measuring  the  residual  error  in 
solving  (1),  and  0(f)  is  a smoothing  norm  which,  in  practice,  is 
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typically  set  equal  to 

n(f)  = (6) 

0 0 

The  success  of  this  procedure  rests  upon  the  assumption  that  the 
desired  solution  is  smooth  (has  several  derivatives) , and  that  any 
errors  in  the  data  supplied  are  random  and  not  smooth.  We  want  the 
regularization  procedure  to  generate  an  approximation  to  our  given 
problem  that 

(a)  is  better  conditioned,  (b)  mollifies  the  effects 

of  any  random  noise  in  our  data,  and  ( c)  gives  us  a 

physically  meaningful  solution  that  approximates  the 

true  solution  in  some  reasonable  sense. 

We  cannot  achieve  this  objective  by  simply  minimizing  the  residual 

error.  The  residual  provides  no  measure  of  the  lack  of  smoothness 

of  a function  f since  the  compactness  of  K may  smooth  oscillations 

in  f.  The  residual  controls  only  the  low  frequency  components  of  f. 

Control  over  the  high  frequency  components  is  exercised  by  the  smoothing 

norm  0 in  (5).  fl(f)  will  be  small,  only  if  f is  smooth.  Thus,  the 

minimization  in  (5)  forces  the  solutions  f of  the  P to  be  smooth. 

a a 

Once  we  restrict  ourselves  to  smooth  functions,  any  such  function  with 
a small  enough  residual  will  be  a good  approximation  to  the  true  solu- 
tion f^.  The  selection  of  a in  (5)  determines  the  tradeoff  between 
these  2 forms  of  control. 

Much  of  the  existing  literature  on  regularization  focuses  on  con- 
structing regularizing  families  and  proving  that  the  required  conver- 
gence occurs.  In  fact  the  implication  in  some  papers  (see  Chechkin  [3] 


for  example)  is  that  the  choice  of  SI  is  a matter  of  convenience, 
and  does  not  critically  affect  the  behavior  of  the  approximations. 

For  kernels  with  singularities,  there  are  results,  such  as  Baglai  [4], 
that  tie  the  choice  of  S)  to  the  type  of  singularity  in  the  kernel. 

We  will  argue,  however,  that  the  choice  of  SI  is  important  even 
for  smooth  kernels  because  numerical  work  demands  more  than  simple 
convergence.  The  major  application  of  regularization  is  to  prob- 
lems where  the  data  is  determined  experimentally  and  contains  noise 
or  errors.  When  there  is  noise  in  the  data,  a in  (5)  cannot  be  made 
arbitrarily  small. 

We  have,  for  any  aSO  the  inequality 

|if(a,e)  - f^ll  < II  f(a,e)  - f(a,0)||  + ||f(a,0)  - f^| | (7) 

where  f^  denotes  the  desired  solution,  f(a,0)  denotes  the  solution  of 
(5)  with  g = g^,  and  f(a,e)  denotes  the  solution  of  (5)  with 

g = g^^  + e for  some  error  function  e . We  note  that  we  use  e to  denote 

both  an  error  function  and  the  norm  of  this  error  function.  Clearly, 
the  size  of  the  first  term  on  the  right-hand  side  of  (7)  is  controlled 
by  the  condition  of  P(a)  and  the  size  of  the  error  e.  If  e »=  0,  as 
a + 0,  this  first  term  will  get  arbitrarily  large.  The  size  of  the 

second  term  depends  upon  the  rate  of  convergence  of  the  P(a)  to  the 

original  problem  P,  as  a 1 0.  Inequality  (7)  clearly  demonstrates 
that  there  are  2 forces  at  work.  To  obtain  a reasonable  approximation, 
each  P(a)  must  be  well-conditioned,  and  we  must  achieve  a reasonable 
rate  of  convergence.  Clearly,  if  the  rate  of  convergence  of  P(a)  to  P 
is  not  fast  enough  so  that  the  noiseless  solution  f(a,0)  of  P(a)  is 
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a good  approximation  to  the  true  solution  for  a 5^  0 and  of  a size 
determined  by  the  noise  e,  then  it  is  unlikely  that  we  will  get  a good 
approximation  to  our  solution  when  there  is  error  in  the  data. 

We  will  demonstrate,  for  certain  classes  of  operators  and  solutions, 
that  the  choice  of  fi  affects  both  the  condition  and  the  rate  of  con- 
vergence. Its  effect  upon  the  condition  is,  however,  of  primary 
importance.  The  condition  of  the  members  of  the  associated  regularizing 
family  improves  as  the  order — the  highest  order  derivative  appearing 
in  fl--is  increased.  Increasing  the  order  of  Q,  generally  has  a negative 
effect  upon  the  rate  of  convergence,  but  this  effect  is  mollified  and 
limited  by  the  smoothness  of  the  desired  solution.  This  is  explained 

I 

in  more  detail  in  sections  3 and  4. 

In  section  2 we  extend  some  results  of  Franklin  15].  He  considered 
3 problems:  (a)  the  analytic  continuation  of  a harmonic  function  that  1 

I 

is  known  on  a circle  of  radius  R < 1 to  the  unit  disk;  (b)  the  back- 
ward heat  equation  in  1 dimension;  and  (c)  the  differentiation  of  a 
smooth  function.  He  showed  that  the  regularizing  families  obtained  for 
the  first  2 problems  by  using  in  (5)  are  poor  approximations  to  the 
original  problem  when  there  is  noise  in  the  data.  However,  the  corres- 
ponding family  obtained  for  problem  (c) , differentiation,  has  reason- 

I 

able  convergence  properties.  He  did  not  elaborate.  In  section  4 we  j 

will  explain  these  results  in  terms  of  the  small  sensitivity  of  the 

condition  of  the  members  of  the  associated  regularizing  families  to 

simple  order  changes  in  the  smoothing  norm  fJ.  This  lack  of  sensitivity 

is  due  to  the  exponential  decay  rate  of  the  singular  values  of  the 


associated  operator.  First,  however,  in  section  2,  we  introduce  norms 
that  yield  regularizing  families  that  are  'good'  approximations  to  prob- 
lems (a)  and  (b)  . 

In  section  3,  we  consider  in  detail  the  problem  of  differentiating 
a function  and  derive  relationships  between  the  rate  of  convergence,  the 
order  of  the  smoothing  norm  f!,  and  the  smoothness  of  the  desired  solu- 
tion. Furthermore,  we  demonstrate  that  the  condition  of  the  associated 
regularizing  approximations  generated  improves  as  the  order  of  Q is  in- 
creased, whereas  the  corresponding  rate  of  convergence  deteriorates. 
However,  the  overall  error  improves.  The  results  derived  in  section  3 
apply  to  a family  of  operators  that  includes  differentiation. 

In  section  4 we  extend  the  results  obtained  for  differentiation 
to  more  general  convolution  equations.  The  results  in  section  4 are 
split  into  5 theorems.  The  first  theorem  yields  majorant  estimates  on 
the  rate  of  convergence  of  Tikhonov  regularizing  families  of  order  p 
for  a large  class  of  convolution  operators.  The  second  theorem  yields 
minorant  estimates  on  these  rates  for  a subclass  of  the  operators  con- 
sidered. Together  these  two  theorems  yield  the  exact  order  of  the  rate 
of  convergence  of  the  regularizing  families  for  these  subclasses.  The 
third  and  fourth  theorem  in  section  4,  again  for  subclasses,  relate  the 
order  of  to  the  condition  of  the  approximating  problems  generated  by 
the  regularization.  As  for  differentiation,  increasing  the  order  of 
improves  the  condition  of  the  corresponding  regularizing  family.  Together 
these  theorems  yield,  for  a subclass  of  the  operators  considered,  an 
overall  estimate  of  the  error  in  the  approximations.  We  conclude 
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section  4 with  a theorem  that  tells  us  that  if  our  operator  K is  sym- 
metric and  positive  definite,  then  there  is  a simplified  regularizing 
family  which  we  can  use  that  is  better  conditioned. 

Sections  3 and  4 use  arguments  which  use  Fourier  transform  analysis, 
similar  to  those  given  in  Aref'eva  |6j.  We  will  discuss  this  paper  as 
well  as  the  earlier  papers  Arsenin  and  Ivanov  [71,  18],  Arsenin  and 
Savelova  [9l,  Goncharskii , Leonov,  and  Yagola  [10]  in  more  detail  in 
section  4. 

In  section  5 we  make  some  remarks  about  several  ways  that  a regular- 
ization can  be  implemented  numerically: 

(a)  Direct  discretization  of  the  functionals  in  (5)  or  of  the 
associated  Euler  equation 

(K*K  + aB)f  = K*g  (8) 

where  B is  the  differential  operator  obtained  from  by  varia- 
tional arguments. 

(b)  Use  of  Fourier  transforms  on  equation  (8) , and 

(c)  Inversion  of  the  differential  operator  B yielding  an  integral 
equation  of  the  second  kind. 

The  discussion  is  not  chronological  since  some  of  the  Russian  work 
predates  Franklin  [5j.  The  results  described  give  us  heuristics  for  con- 
volution equations  that  can  be  used  in  estimating  the  appropriateness  of 
a proposed  regularization. 


? ■ 
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2.  Franklin  [5.1 

In  this  section  we  examine  the  2 classical  ill-posed  problems  con- 
sidered by  Franklin  [5]: 

(a)  The  analytic  continuation  of  a harmonic  function  in 
the  unit  circle  (see  equation  (17))  and 

(b)  The  solution  of  the  backwards  heat  equation  in 
1-dimension . 

Theorem  1 . Franklin  f5].  For  the  regularizing  families  obtained  for 

problems  (a)  and  (b)  above,  using  the  norm  in  (6)  and  assuming 
2 2 

d^e  Sai'd^r  , one  obtains  the  following  estimates  of  the  error  in  the 
approximation  of  the  desired  solution  f^  by  the  solution  f(a,e)  of  P(a) 
with  + c, 

(a)  I |f  (a,e)  - foil  S I C^/(-«na)  ] (9) 

(b)  ||f(a,e)  - foil  ^ fT/(-i(na)  1^^^.  (10) 

In  (10)  [0,T]  is  the  interval  of  interest  for  the  heat  equation,  and  this 
result  was  obtained  with 


We  note  that  the  setting  in  Franklin  [5]  is  different  from  what  we 
will  use  in  later  sections.  Franklin  considered  varying  a and  c simul- 
taneously. He  asked  the  question:  If  ||ku||se.  and  fi(u)sl,  then  how  does 
the  ||u|l  behave  as  eVO?  He  called  this  the  modulus  of  regularization. 
In  his  arguments,  he  proved  that  if  we  consider  functions  u such  that 
(i(u)SQ,  then  using  2 choices  for  e and  Q he  can  bound  the  ordinary 


9 


rrodulus  of  convergence  by  multiples  of  the  modulus  of  regulariza- 
tion. Thus,  estimating  the  modulus  of  regularization  yields  a corres- 
ponding estimate  on  the  modulus  of  convergence  as  c and  a decrease  to 
zero.  The  estimates  in  (9)  and  (10)  were  obtained  using  the  modulus 
of  regularization  and  assuming  that  d^c'^Said^c  . The  Franklin  argu- 
ments (5]  used  Fourier  series  expansions  of  the  solutions  of  the 
(a)  analytic  continuation  and  (b)  the  backwards  heat  equation  prob- 
lems. In  this  section  we  will  use  similar  arguments  to  derive  error 
estimates  for  other  norms.  The  class  of  problems  considered  in  section  4 
will  include  these  2 problems.  However,  we  also  consider  them  separately 
here  because  Franklin's  arguments  (5  I explicitly  relate  the  control 
exercised  by  the  residual  term  in  (5)  to  the  low  frequency  components 
of  the  solution  and  the  control  exercised  by  the  smoothing  norm  (1  in  (5) 
to  the  high  frequency  components  of  the  solution.  In  section  4 we  will 
demonstrate  using  Fourier  transform  analysis  that  the  high  frequency  norm 
control  is  the  ii.ullifier  of  the  condition  of  the  regularizing  approxima- 
tions . 

If  we  know  for  problem  (a)  that  for  the  desired  solution  the  quantity 

(f)  = F I |f  |^c^V(2j)  I (12) 

j=0 

exists  and  is  finite  for  some  c>0,  where  the  L^-norm  is  over  the  interval 
of  interest,  then  we  can  use  fi^  to  define  a regularizing  family  for  problem 
(a) . We  will  demonstrate  that  for  c2-2KnR,  the  regularizing  family 
generated  by  £1^  yields  'good'  approximations  to  problem  (a). 
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Lemir.a  1 . Let  f be  C on  and  assume  as  |t|-x»  for  j=0,l,2.. 

Assume  also  that  the  Fourier  transform  of  f exists  and  satisfies 

l^(w)  I < M exp(-vi  |w|  ) (13) 

for  all  1 0. 1 >S  and  for  some  M>0.  Then,  if  2(j>c,  il^(f)  is  finite. 

Proof . 

By  (13)  the  Fourier  transform  of  each  derivative  of  f exists  and 
is  in  . By  Parseval's  equality, 

(14) 

where  the  co-norm  is  on  (-“,“>)  . 

Therefore,  each  term  in  (12)  is  majorized  by 

(M/{2j)!)  ^ (coo)  ^^exp  (-2y  I 0)  I ) < M (c/2y)^^ 


(15) 


Combining  (15) , (14)  and  (12)  we  obtain 

cc 

Sl‘=(f)  < M [ (c/2y)^^. 
j=0 

But,  this  series  is  finite,  whenever  2y>c.  Q.E.D, 

Theorem  2.  If  for  the  solution  f^  of  the  analytic  continuation  problem 

c c 

(fQ)<“,  then  for  the  regularizing  family  defined  in  (5)  using 

in  (12) , we  obtain  the  following  majorant  estimate  of  the  error  as  a+O, 


I f (a,E)  - f 


O' 


i Q*a 


c/ (c-2HnR) 


(16) 


2 2 

We  have  assumed  that  d^^e  fa^d^c  . 


Proof.  We  repeat  Franlclin's  analysis,  using  instead  tlie  H - norm.  We  are 

00 

given  a harmonic  function  u(r,6)  in  the  unit  dis)c  with  )uiown  values  for 
some  R<1,  g^*  u(R,0),  and  as)ced  to  determine  its  values  fQ«=u(l,0)  on 
the  unit  circle.  We  must  solve  the  integral  equation  of  the  first  )(lnd, 
g^^  - l/2it  /^’’f  1-R^/l-R  cos(e-ej)+R^.l  . 


(17) 
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V.'e  know  the  solution  is  periodic  so  we  write  the  error  h^=f(a,e)-fp  as 

h (0)  = y (A  cosn6  + B sinn0)  . 
a n n 

n=0 

If  j l^h^l  l^'^aQ  and  obtain,  assuming  that  the  interchanges  of 

summation,  integration,  and  differentiation  are  correct, 

,c„  . ..  .2  . r , , . > ,-2  . 


n (h  ) -■  2tiA  + tt  y cosh(cn)  (A  + B ) < 20 
a 0 1 n n 

“ n=l 


(IP) 


and 

OO 

I l^h^l  = 2tiAq  + tt  ®n^  ~ 

n=l 

Following  Franklin,  we  balance  the  contribution  from  the  low  frequency 
terms  in  (19)  against  the  contribution  from  the  high  frequency  terms  in 
(18)  by  choosing  an  appropriate  value  for  n.  Let  N=N(a)  be  the  solution 
of  the  equation 

2N 

acoshcN  = 2R  . (20) 

Since  for  ns'N,  and  for  n>N,  coshcn>coshcN , we  obtain 

I )h  11^  < IlKh  II^R  + 0(h  ) (coshcN)  ^ < 2p(coshcN)  (21) 

' ' a ' ' a a 

Now  consider  the  asymptotic  behavior  of  N as  a function  of  a.  Clearly, 
as  a + 0,  N -t",  and 


N '\-(l/(c-2«nR)]  £n(4/a)  . (22) 

Substituting  (22)  into  (21)  we  obtain  (16)  . O.  E.  D. 

Comments . If  c=  -2)f.nR,  then  the  exponent  in  (16)  equals  1/2,  For  fixed 
R,  as  we  let  c+“  this  exponent  approaches  1.  As  we  decrease  c,  it  de- 
creases to  0.  Note  that  the  eigenvalues  of  K*K  decay  like  exp  (2nHnR)  as 
nt“>.  Choosing  a norm  with  a growth  rate  that  matches  or  exceeds  this 

decay  rate  yields  a reasonable  rate  of  convergence.  To  use  we  must 

00 

know  that  for  the  desired  solution  fi  (f.^)<“. 

OO  ' 0 
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We  can  achieve  a better  rate  of  convergence  if  we  use  an  even 
stronger  norm. 

= I |f 1 1^/jl  for  c>0  (23) 

j=0 

Then  we  have  the  follov.’ing;  the  proof  is  similar  to  that  of  Lemma  1. 

00  ( T ) II 

Lemma  2 . Let  fcC  on  and  assume  f (t)-*-0  as  |tJ-»«  for  all 

j=0,l,2...  Assume  also  that  the  Fourier  transform  of  f exists  and 
satisfies , 

|^((d)|  S M exp(-uw^)  (24) 

for  all  |u)|>S,  for  some  M>0  and  \i>c/2.  Then  (f)  is  finite. 

Theorem  3 . If  for  the  solution  f^  of  the  problem  of  analytic  continuation 
fi  for  some  c>0,  then  for  the  regularizing  family  defined  in  (5) 

Oo  Q 

using  in  (2  3),  we  obtain  the  following  majorant  estimate  of  the  error 

cc 

2 2 

in  the  approximation  as  a,e4-D  with  # 

I |f  (a,e)  - fpl 1^  < Q*a  . (25) 

This  estimate  is  also  valid  for  general  a and  R near  1. 

Proof . The  proof  proceeds  as  before  with  small  changes.  Equation  (20) 
becomes 

2 2N 

a exp(cN  ) = 2R  . (26) 

The  asymptotic  solution  of  this  satisfies 

1/2 

N -V  (-in(a/2)/c)  . (27) 

for  a given  R and  sufficiently  small  a,  or  for  a given  a and  suffi- 
ciently large  R.  Then  using  (27)  and  the  analog  of  (21) , we  obtain  (25) . 

Q.E.D. 


tr-.ixw..  X 
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c — c 

Comments.  We  note  that  to  use  n or  fj  numerically  we  would  have  to 
use  Fourier  transforms,  and  we  would  have  to  know  that  our  desired  solu- 

C “C 

tion  had  sufficient  smoothness  to  guarantee  the  existence  of  or  SI  . 

00  CO 

We  note  that  as  R4'0,  the  eigenvalues  n=l,2,...  of  the  operator  K 
in  (17)  decrease  more  rapidly,  and  the  error  in  the  approximation  in- 
creases. Thus,  by  picking  a norm  whose  growth  rate  matches  or  exceeds 
the  decay  rate  of  the  given  operator,  we  can  achieve  'good'  orders  of 
approximation . 

If  we  apply  the  same  analysis  to  the  backwards  heat  equation  we 
obtain  the  following. 

Theorem  4 . If  for  the  solution  f^  of  the  backwards  heat  equation  in 
1-dimension  we  have  then  the  regularizing  family  defined 

in  (5)  using  in  (23)  satisfies  the  following  majorant  estimate 

I |f(a,E)  - fgl  1^  < (2£ 

The  heat  equation  is  considered  on  the  interval  (0,T) , and  again  we 

2 2 
assume  d^e  Sosd^e  . 

Proof.  In  this  case  we  obtain  the  equation 

a exp(N^(c  + 2T) ) =2  (2£ 

Taking  logarithms  of  both  sides  we  obtain. 


Therefore , 


N (c  + 2T)  = -In (a/2)  . 


N = -l!,n(a/2)/(c+2T) 


||h  11^  s Q exp(-cN^)  <: 


Q.E.D. 
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c 

Theorem  5,  If  we  use  51  , we  obtain  an  estimate  of  the  form 

» I " — — — . . (JO 

||h^||^  < p*  exp(-c (-Una) (30) 
where  c involves  c and  T. 

The  difference  in  the  convergence  of  the  regularizing  families 
for  problems  (a)  and  (b)  can  be  explained  by  the  difference  in  the  decay 

rates  of  the  eigenvalues  of  the  2 operators.  For  problem  (a)  the  eigen- 

n 2 c 

values  decay  li)ce  R ; for  problem  (b)  li)ce  exp(-n  T)  . Thus,  the  norm  51 

OO 

with  a growth  rate  of  exp(cn)  cannot  counteract  the  rapid  decay  of 

exp(-n^T).  For  problem  (b)  we  must  go  to  whose  growth  rate  is 
2 

exp(cn  ),  to  achieve  an  overall  approximation  that  is  some  power  of  a. 

Tlie  preceding  analysis  points  out  the  need  to  have  an  estimate  of 

the  decay  rate  of  the  given  operator  K as  an  aid  in  choosing  an 

appropriate  51.  We  are  of  course  constrained  in  what  order  51  we  can  use 

by  the  smoothness  of  the  desired  solution.  In  order  for  the  convergence 

arguments  to  be  valid,  ^^(fg)  must  be  defined. 

Arguments  of  the  preceding  type  assumed  a and  c varied  simultaneously 
2 2 

and  in  fact  that  d^^e  ^as^d^e  . Consequently,  the  effect  of  51  on  the  indi- 
vidual rates  of  convergence  and  on  the  condition  of  the  approximating 
problems  cannot  be  obtained  from  such  arguments.  In  sections  3 and  4 we 
attempt  to  estimate  directly  rates  of  converqence,  and  the  condition  of 
members  of  the  Tilthonov  regularizing  families  of  more  general  convolution 
operators . 

In  the  next  section,  we  examine  the  problem  of  differentiation  in 
detail.  We  will  be  using  Fourier  transforms  so  instead  of  equation  (1) 
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we  use 

f^f(s)ds  = g(t)  -oo<t<«> 

^00 

where  we  have  assumed  g(-'»)=0. 

We  will  show  that  increasing  the  order  of  makes  the  rate  of  con 
vergence  deteriorate,  but  improves  the  condition  of  the  approximating 
problems  generated.  The  net  effect  is  to  improve  the  overall  error 
I I f (ajej-f^l  I if  a<Ce^  for  some  C. 


(31) 
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3.  Differentiation. 

Now  let  us  use  differentiation  to  understand  the  effect  on  the  rate 
of  converge'  ce  and  on  the  conditioning  of  the  regularizing  approximations 
for  various  choices  of  the  norm  12.  We  want  to  demonstrate  the  interplay 
between  the  choice  of  12,  the  rate  of  convergence,  the  conditioning,  and 
the  smoothness  of  the  desired  solution.  We  will  work  on  the  interval 
(_t»,oo)  using  equation  (31),  and  use  Fourier  transform  analysis.  In  fact, 
the  following  theorems  are  valid  for  any  operator  K with  Fourier  transform 
|ic(u)  I = r>0  . (32) 

This  includes,  in  particular,  all  the  derivative  operators.  We  use  argu- 
ments similar  to  those  in  Aref'eva  1 6 ]. 

Theorem  6.  We  assume  that  the  desired  solution  f^  has  m>l  derivatives, 

that  each  f^^^  e (-“,“>)  O^jfm,  and  its  Fourier  transform  t satisfies 

for  all  CO  and  |?|>D  co  ^ for  large  co.  Then  for  any  equation 
j+1  J- 

(1)  whose  kernel  K satisfies  (32),  we  have  the  following.  For  sufficiently 
small  a,  the  rate  of  convergence  of  the  Tikhonov  regularizing  families 
obtained  using  f2^  in  (11)  , for  mS2p+2r-l/2  varies  as 

||f(a,0)  - foil  ^ where  a = • (33) 

For  m>2p+2r-l/2  this  rate  is  at  least  0(a).  The  dynamic  condition  of  the 
members  of  these  families  varies  as 

, where  b = -r/(p+r)  (34) 

Moreover,  for  any  m&l , p?0,  and  r>0,  an  overall  estimate  of  the  error  in 
using  the  solution  of  P(a)  with  g^*gQ+c  to  approximate  f^  is  of  the  form, 

I |f(a,E)  - fgl  I < C^a'’  + 


(35) 
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where  b=-r/(p+r)  and  q= (2m+l ) / (4p+4r)  if  m<2p+2r-l/2 , and  q>l  otherwise. 
Comments . The  smoothness  of  the  solution  helps  the  rate  of  convergence, 
but  of  course  does  nothing  for  the  error  term.  Using  (35) , we  see  that  in- 
creasing the  order  p of  SI  adversely  affects  the  rate  of  convergence,  but 
has  a significant  effect  on  the  condition,  which  typically  would  outweigh 
in  (35)  any  negative  influence  on  the  rate.  As  we  increase  the  decay  rate 
r of  K,  the  condition  and  the  rate  deteriorate.  As  r the  estimate  of 
the  condition  approaches  1/a  and  of  the  rate  approaches  0(1).  If  we  fix 
r and  increase  the  order  p,  the  convergence  rate  estimate  in  (33)  approaches 

a®  where  e = (2m+l)/(4m+4r)  (36) 

since  psm;  and  in  (34),  the  exponent  b approaches  -r/(m+r).  We  show  in 
Theorem  7 that  in  fact  the  optimal  estimate  in  (35)  is  achieved  when  p=m. 

By  rate  of  convergence,  we  will  always  mean  the  size  of  the  L^-norm 
of  the  discrepancy  caused  by  the  regularization,  h^=f (a,0) -f^. 

By  condition  we  mean  the  dynamic  condition  Ma)thoul  fllj  which  is  a 
measure  of  the  difficulty  of  the  deconvolution  of  the  associated  Fourier 
transform  problem.  In  cases  where  the  discrete  approximation  used  is  a 
symmetric  Toeplitz  matrix,  the  algebraic  condition  of  these  matrices  is 
not  worse  than  that  given  by  the  dynamic  condition. 

Proof  of  Theorem  6.  First  consider  the  rate  of  convergence  of  the  asso- 
ciated regularizing  family  as  a function  of  p,  m,  and  r.  Consider 
given  by  (11)  for  p=l,2...  We  use  Fourier  transforms  and  the  Euler  equa- 
tion (8)  to  compute  f^.  Then  the  discrepancy  caused  by  the  regularization 

I 

i V 
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h^=f  (u,0) using  Parseval's  equality,  satisfies 
I Iv,  I . .2  r ^ 4p+4r  A 2 


<37) 


0 — 2p+2r.2 

(c+aoj  ) ir 


Part  a:  Rate  of  convergence  - Majorant  estimate 

We  split  this  integral  into  the  sum  of  integrals  over  (0,M)  and 

where  M will  be  chosen  subsequently  to  optimize  the  error  estimate.  If 


have  then  we  have  for  any  M,  and  0<j^m+l 


M 


M 


= P.M 
1 


Moreover,  for  0< j <J=min (2p+2r+l/2 , m+1) 
M „ „ M 


/■[ft  /■■  ^4p+4r-2j  _ „,^2  „4p+4r-2j+l 


= B*a  M 
1 


(38) 


(39) 


0 ^ 0 

Combining  (38)  and  (39) , we  obtain  the  majorant  estimate  for  any  jSJ  of  the 


form. 


|h  ||2  < + Ey-2^ 

' a ' ' 1 2 


(40) 


If  we  choose  M to  minimize  the  expression  in  (40) , we  obtain  for  each 
such  j , 


M,  = Co 
D 


-l/(2p+2r) 


(41) 


or  a ■»  CM  (2p+2r) ^ Thus,  the  optimal  choice  of  M is  independent  of  jSJ, 


and  we  obtain  the  optimal  majorant  estimate  obtainable  from  (40)  as 

llh 


< Qj“*  with  a = (2 j-1) / (2p+2r) 


(42) 


where  depends  on  j but  not  on  a.  In  particular  for  a given  r and  p, 
if  the  smoothness  of  f satisfies  ms2p+2r-l/2,  then  the  best  order  of 
a in  (42)  corresponds  to  j=m+l  for  which  we  obtain  (33).  If  m>2p+2r-l/2, 
then  the  best  order  is  obtained  with  j»=2p+2r+l/2  when  we  obtain 


IMI' 


5™^ 


(43) 
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Note . The  constant  terms  in  the  various  equations  do  not  involve  a and 
do  not  significantly  affect  the  arguments  so  no  attempts  have  been  made 
to  be  precise  with  them.  We  note,  moreover,  that  the  estimates  in  (42) 
and  (43)  hold  for  all  a>0. 


Part  b:  Rate  of  Convergence  - Minorant  Estimate 


We  must  obtain  a lower  bound  on 


Since 


-m-1  - , 

w for  large 


w and  |k|=Du)  ^ everywhere,  we  have  for  sufficiently  small  a that 


2 ^ / 2 4p+4r-2m~2 

> C r.  aw* 


M 


,,  2p+2r,2 

(1+aw  * ) 


where  M is  defined  in  (41)  . But  for  w>M  we  have 

2p+2r  ^ „ 
aw  >00  . 

Therefore,  we  have  with  Q=(C+1)/C,  on  (M,“)  the  denominator  in  (44) 
satisfies , 

,1  2p+2r,  . . 2p+2r 

(1  + aw  ) s Qaw 


(44) 


(45) 


(46) 


From  (46)  we  obtain 


^ ! -2m-2 

^ 0 ..  w = 


Q*M 


■2m-l 


Therefore,  using  (41)  we  obtain  for  sufficiently  small  a that 
||h^||^  ^ Qa®  with  a = (2m+l)/(2p+2r)  . 


(47) 


The  coefficient  Q in  (47)  depends  on  p,  m,  and  r but  not  a. 

Combining  (33)  and  (47)  for  m^2p+2r-l/2,  we  obtain  an  exact  order  esti- 


mate of  the  rate  of  convergence 

||h^||^  < O^a®  with  a = (2m+l) / (2p+2r)  . 

For  m>2p+2r-l/2,  we  get  that  the  convergence  is  at  least  as  fast  as  a^ 


(48) 


|h  ir  ^ OV 


(49) 
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r. 


rnrt  c.  Conditioning 

The  condition  of  a minimization  problem  typically  refers  to  the  con- 
dition of  the  second  derivative  Of'Crator  of  the  qiven  functional.  In  our 
case  this  is  the  operator  in  the  Euler  equation  (8) 

({.  = K*K  + aB  (50) 

where  B is  qenerated  from  fi  using  variational  arguments. 

For  differentiation, 


f + af  - af  . (51) 

t 0 

If  we  approximate  this  operator  and  f by  discretizing,  then  we  obtain  a 
matrix.  However,  the  type  of  matrix  obtained  Toeplitz,  symmetric  etc. 
depends  upon  the  quadrature  rule  used  for  K*K  and  the  approximation  of 
the  differential  operator  chosen.  The  algebraic  condition  may  vary  de- 
pending upon  the  particular  approximation  chosen.  Therefore,  we  choose  to 
loo)c  at  this  operator  directly  and  to  use  Fourier  transform  analysis. 

We  use  the  dynamic  condition,  Ma)choul  111],  the  maximum  and  minimum  values 
of  the  modulus  of  the  Fourier  transform  of  our  given  operator.  This  notion 
of  conditioning  is  only  applicable  to  convolution  operators. 

Therefore,  we  form  the  Fourier  transform  of  i)> , and  consider 


h((j)  = U (ij) 


-2r  2p 

u)  + au) 


(52) 


In  (52)  we  have  ignored  the  lower  order  derivative  terms  in  because  tliey 
have  only  secondary  effects  upon  the  conclusions.  The  minimum  of  h occurs 
at  io=(r  ^and  is  of  the  order 

Ca^, 


a = r/ (r+p)  . 


(53) 
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(53)  is  a lower  bound  on  the  smallest  eigenvalue  of  the  operator  ip . Observe 
that  < has  no  maximum.  However,  in  numerical  wor)c , there  ’ s a limit  on  the 
sampling  size,  and  the  maximal  singular  value  of  any  discretization  of 
depends  primarily  on  this  sample  size.  To  see  this  consider  the  following. 
We  )<now,  see  e.g.  Wilkinson  [12],  that  for  any  matrix  A that  approximates  K, 
all  the  eigenvalues  of  A satisfy  | X ^ (A) | < | | A | | for  any  consistent  matrix 
norm.  In  particular  the  maximal  row  sum  norm  is  a bound  on  the  eigenvalues. 

So  we  can  conclude  that  as  the  order  of  Q is  increased,  the  condition 
of  the  approximating  problems  improve,  and  the  critical  order  in  a is  given 
in  (34). 

Tart  d.  Overall  Estimate  of  Error  in  the  Approximation 

Now  let  us  consider  the  overall  error  in  (7) . Using  Fourier  transforms 
we  find  that  the 

I |f  (a,e)  - f(a,0)  | |^  = | |^(a,e)  - ?(a,0)  | S with  b = -r/(2r+2p)  (54) 

Combining  (54),  (48)  and  (52)  we  obtain  (35).  Q.E.D. 

Theorem  5 tells  us  that  for  small  decay  rates  r we  can  achieve  fairly 
well-conditioned,  approximating  problems  with  a small  order  norm  SI.  For 
computing  the  first  derivative  of  a function  we  have  r=l . Therefore,  for 
m*-2p+3/2  we  have  the  convergence  rate  is  of  order 

with  a = (2m+l)/(2p+2) 
and  the  condition  is  bounded  by 

C«^  with  b = -l/(p+l)  . 

Therefore,  for  a given  m and  p>m/2-3/4,  we  have  that  the  rate  becomes  worse 
as  we  let  ptm.  At  p=m  the  convergence  rate  is  nearly  1.  The  condition  im- 
proves significantly  as  p+m,  its  best  value  is  achieved  when  p=m. 
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We  have  the  following  simple  theorem. 

Theorem  7 . Under  the  hypotheses  of  Theorem  6,  for 

a<{C^t/C^)^  (55) 

and  ms2p+2r-l/2,  the  choice  yields  the  regularizing  family  that  minimizes 
the  overall  error  in  (35) . 

Proof  of  Theorem  7.  We  can  approximate  the  minimum  of  (35)  by  solving  the 
equation  oi^=ec^/c^  where  b= (m+l/2+r ) / (2r+2p)  for  p since  the  first  term  in 
(35)  is  a monotone  increasing  function  of  p and  the  second  a monotone  de- 
creasing function.  If  the  solution  lies  in  the  interval  p>m  then  the  mini- 
mum value  corresponds  to  p=m.  Ta)cing  logarithms  of  both  sides  of  this 
equation  and  using  (55)  we  get  the  solution  p»  satisfies 
2p*  + 2r  > 2m  + 2r  + 1 . 

Therefore,  the  optimal  choice  is  p=m.  Q.E.D. 

In  the  next  section  we  consider  more  general  convolution  operators, 
again  on  the  interval  (-“>,")  . 


1 


* « <u 
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4.  General  Convolution  Equations. 

In  this  section  we  extend  the  results  obtained  in  the  previous  section 
for  operators  of  the  form  | K (w) | =Cio  r>0,  to  more  general  convolution 
equations.  Arsenin  and  Ivanov  1.7],  Arsenin  and  Savelova  (9  I,  Goncharski, 
Leonov  and  Yagola  TlO]  and  Aref'eva  I6J  have  each  derived  estimates  on  the 
asymptotic  rates  of  convergence  (as  alO) , and  on  the  overall  error  bounds 
attainable  when  the  data  is  inaccurate,  for  certain  classes  of  convolution 
equations  on  the  interval  Each  paper  used  Fourier  transform  analysis. 

The  families  considered  in  Aref'eva  ,6.1  will  be  described.  These  include 
those  considered  in  the  other  papers.  The  papers  are  listed  chronologically. 
The  Arsenin  and  Ivanov  [7]  and  Arsenin  and  Savelova  [9]  papers  obtained 
majorant  estimates  that  relate  the  decay  of  the  transform  of  the  operator  K, 
the  order  of  the  norm  Q,  and  the  smoothness  of  the  desired  solution.  They 
used  complex  variable  theory  to  evaluate  the  error  integrals.  Goncharski, 
Leonov  and  Yagola  [lOj  enlarged  the  class  of  kernels  under  consideration  and 
used  the  L^-norm  of  the  Fourier  transform  of  the  error  so  they  could  work 
with  real  integrals.  However,  they  did  not  consider  general  norms  SI  or  the 
effects  of  the  smoothness  of  the  desired  solution.  Aref'eva  I 6]  considered 
the  same  class  of  kernels  as  in  (lOj.  However,  he  answered  a slightly 
different  question.  Generalizing  the  class  of  admissible  regularizers , he 
obtained  the  following  Theorem.  For  a precise  statement  see  Aref'eva  [6J. 
Theorem  8.  Aref'eva  [bl.  For  a given  level  of  noise,  a,  the  optimal 
approximation  f^  to  the  desired  solution  of  (1)  is  obtained  with 


where  B is  the  transform  of  the  operator  B in  (8) . 
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Thus,  in  the  presence  of  noise,  the  optimal  choice  of  reijularizer 
depends  only  on  the  decay  rate  of  the  Fourier  transform  of  the  desired 
solution.  We  note  that  the  expression  ir.  ‘‘•w  may  or  may  not  have  a function 
analytic  interpretation  as  the  norr  cf  some  . ient i f iable  function  space. 

All  Tikhonov  regularizers  are,  of  course,  defined  initially  in  terms  of 
such  norms.  Using  (56),  Aref'eva  [6!  derives  estimates  on  the  optimal 
error  achieved  in  terms  of  o.  Since  (56)  is  not  a priori  computable, 

Aref'eva  then  returns  to  Tikhonov  regularizers  and  shows  that  order-wise, 
for  certain  classes  of  problems,  he  can  achieve  to  the  same  order  the 
error  that  he  gets  using  the  optimal  regularizer  in  (56) . For  certain 
subclasses  he  obtains  order  exact  estimates. 

In  this  paper  we  consider  only  Tikhonov  regularizers.  When  we  make 
this  restriction  then  the  character  of  the  compactness  of  the  operator  and 
the  smoothness  of  the  desired  solution  are  both  of  primary  importance.  The 
character  of  the  compactness  of  K determines  whether  or  not  the  approxima- 
tion obtained  using  a particular  family  of  regularizers  is  sufficient  to 
j make  sense  computationally.  As  we  saw  in  section  2 with  exponentially 

decaying  operators  K,  in  order  to  obtain  an  approximation  of  the  order 

for  some  a>0,  we  had  to  use  an  exponentially  increasing  smoother  (2  . 

00 

C . 

Obviously,  the  use  of  fl  is  limited  by  the  smoothness  of  our  desired  solu- 
; tion.  The  convergence  arguments  are  valid  only  when  is  defined. 

Numerically,  we  have  to  worry  about  the  condition  of  the  approximating 


problems  generated.  As  in  section  3,  we  will  use  the  dynamic  condition. 
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The  discussion  has  3 parts.  First  for  a large  class  of  operators  we 
obtain  in  Theorem  9 a majorant  estimate  of  the  rate  of  convergence  of  the 
approximating  regularizing  feimily  when  there  is  no  noise.  In  Theorem  10 
for  a subclass  of  these  operators  we  obtain  a minorant  estimate  of  this 
rate.  Together  Theorems  9 and  10  give  us  the  order  of  the  rate  of 
convergence  for  this  subclass.  In  Theorems  11  and  12  for  a further  subclass 
we  estimate  the  dynamic  condition  of  the  approximating  problems  generated 
using  (Theorem  11)  and  using  and  51  (Theorem  12) . Together  these 

P 00  oo 

theorems  give  us  an  estimate  on  the  overall  behavior  of  the  error  in  approx- 
imating our  given  solution  f^  by  the  solution  of  P(a)  for  g=g^+E . 

Using  these  estimates  we  can  argue  that  typically  one  would  obtain 
the  best  error  by  maximizing  the  order  of  51.  For  a function  f^  with  m 
derivatives  this  corresponds  to  taking  p=m.  We  note,  however,  that  in  prac- 
tice we  often  do  not  know  m and  we  are  likely  to  choose  a low  order  51  be- 
cause it  is  easier  to  work  with.  In  such  a situation  it  is  important  to  have 
some  estimate  of  the  decay  rate  of  our  operator.  Since  we  are  nominally 
working  with  symmetric,  Toeplitz  operators  the  associated  Fourier  transforms 
give  us  such  an  estimate.  Using  this  estimated  decay  rate  we  can  use  the 
inequalities  obtained  to  indicate  whether  the  proposed  regularization  makes 
sense  numerically. 

The  following  are  slight  modifications  of  definitions  in  Aref'eva  [6]. 
Definition  2.  Aref'eva  [6].  A kernel  K is  of  type  1 if  (a)  Its  transform, 
|A(o))  ( has  a denumerable  number  of  zeros  on  the  real  axis  of  u.  (b)  There 
are  no  limit  points  for  the  zeros  of  A(oj)  with  the  exception  of  w=+“>,  and  if 
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V 

N(M)  is  the  number  of  zeros  in  (0,M),  then  N(M)<CM  for  some  vSO,  C20; 

(c)  There  exists  a system  of  non-overlapping  e-neighborhoods  of  the  zeros 
of  K(u))  on  wSO  such  that  | K (w)  I >a  . I w-w . I ^ for  wel  oi . -e.>) . +c  . ] where  the 

1 1 1 L 1 1 

are  the  finite,  nonzero,  zeros  of  K and  a^^a>0,  s>0.  (d)  If  1^(0) 

then  there  is  an  interval  10, e^)  in  which  |k(u)|>C  . (e)  Outside  these 

e-neighborhoods,  |k(w) |^D  w r>0,  D >0. 

We  note  that  K can  have  infinitely  many  zeros  satisfying  (c)  only 

if  e.->-0  as  i-»«>. 

1 

Definition  3.  Aref'eva  [6l.  A kernel  is  of  type  2 if  for  large  w 
|k(w)  I^C  exp(-Ao))  for  some  A>0,  C>0. 

Within  type  1 Aref'eva  [6]  determines  a subclass  for  which  he  also 
derives  a lower  bound  estimate  of  the  same  order.  Aref'eva  (.6]  also  class- 
ifies solutions  in  terms  of  the  behavior  of  their  Fourier  transforms. 

He  further  subdivides  type  1 into  0<s<l/2,  integrable  singularities  and 
s>l/2,  nonintegrable  singularities.  We  consider  only  s<l/4.  We  use 
Aref'eva  type  arguments  to  derive  our  estimates  in  Theorems  9 and  10. 

We  note  that  the  arguments  in  section  3 also  mimic  Aref'eva  [6]  arguments. 
Theorem  9.  Consider  the  norms  p>l  and  equation  (1) . Let  the  solution 
fg  of  (1)  have  m21  derivatives  with  each  f^^^  , 0<j<m  and 

|io^  ^^(u)|<R_f^r  all  iL).  (a)  Let  K be  of  type  1 with  0Ss<l/4  and  vS4s. 

Then  the  discrepancy  h^=f(a,0)-fQ  satisfies  the  following. 

For  p i m/2  - r + 1/4, 

||h^||^  S with  b«  (2m+l ) / (2p+2r) 

and  for  p < m/2  - r + 1/4  , I |h  1 I S C_a  . 

a 2 


(57) 


27 


(b)  Let  X be  of  type  2 with  | ft 1 2Dexp  (-aw) for  some  a>0  and  for  all  w.  Then 

(B8) 


|hj  < C3(-ina)-<^"'''^> 


In  (57)  p is  the  order  of  n,  r is  the  decay  rate  of  |k| , and  s 
and  V are  as  specified  in  definition  2. 

Proof  - Part  a.  (The  argument  parallels  that  in  Theorem  2 of  Aref'eva  L6l.) 

The  proof  is  analogous  to  that  given  for  Theorem  5,  using  expression  (37) 

for  I |h^| I except  that  it  is  now  necessary  to  split  the  integral  over 

(0,M)  into  integrals  over  neighborhoods  of  the  zeros  of  ft.,  and  integrals 

over  the  complements  of  these  neighborhoods.  For  each  neighborhood 

N . = (w . -e . ,w. +e . ) of  a zero  w.  of  |k|  we  have 
11111  1 ' ' 

/ * c»V-''= 


N. 

1 


N,  I |4s 
1 |w-w^| 


There  are  at  most  M such  intervals  in  (0,M)  and  clearly  each  e^<M,  so 


we  obtain 


I / |(i s 

i N, 

1 

Now  consider  the  integrals  over  the  complements  of  the  . On  the 
complements  |ft|>Dw  ^ 

I i |ft  < Da^  ^ 

N.  “ 0 ^ 

1 

for  any  j<J=min(m+l,  2p+2r+l/2) . 

The  integral  from  (M,“) , as  in  Theorem  5,  satisfies  (44).  Therefore, 
1 2 . 2-  y ^ ^ ^1-2 j 


|h  I I S a'^CM^  + D.M'^ 
' a ' ' j 


(59) 


where  u=max (l+v-4s , 4p+4r-2j+l) . Minimizing  this  estimate  over  M we  obtain 

-l/iT 


M _ = C„a 
opt  0 


(60) 
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where  u=max (v/2-2s+ j , 2p+2r) . We  note  that  p=2p+2r  if 

j < 2p  + 2r  + 2s  - v/2  . (61 

But  we  know  j<2p+2r+l/2,  so  if  4s>v,  then  (61)  is  satisfied,  and  we  have 


for  all  j^J  that 


M „ = C.a 
opt  0 


-l/(2p+2r) 


Therefore,  for  p^/2-r+l/4  we  obtain  (57)  and  for  p<m/2-r+l/4  the  error 

2 

decreases  at  least  as  fast  as  a . 

Proof  - Part  b.  Exponentially  decaying  1 | SDexp  (-aui)  for  some  a>0, 
and  all  u).  We  have  assumed  that  ic  has  no  real  zeros.  We  again  split 
the  error  estimate  (39)  into  integrals  over  (0,M)  and  (M,“) . 


/ ^ I ^ a^C^e' 


0 0 „2  2 4p  4a(u 

D +a  u)  ^e 

Moreover,  we  obtain  (44)  again. 


Combining  (44)  and  (62)  and  then  minimizing  over  M,  we  obtain  the 

transcendental  equation  for  the  optimal  M 

^ „ 2 4aM  ^ ..-2m-2 

4aC^a  e = C^M 

Taking  logarithms,  we  find  that  the  solution 

M*  'V-  - fn(ba^)/4a  - (An((-JlnbaV4a))(m+l)/2a  (6 

where  b=4aCj^/C2.  Therefore,  a majorant  estimate  is  of  the  form 

I |h^|  1^  ^ C (-4n(ba^))"^^'"‘*'^’  . (( 


Comments,  Part  (a)  of  Theorem  9 states  that  we  can  obtain  for  a larger 


class  of  operators  the  same  majorant  of  the  rate  of  convergence  that  was 
obtained  in  Theorem  5 as  long  as  the  growth  rate  v of  the  number  of  zeros 
in  the  interval  (0,M)  is  not  greater  than  4s.  The  exponent  s in  defini- 
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tion  1 is  a measure  of  the  rate  of  decrease  of  ft  in  a neighborhood  of  any 
zero.  Part  (b)  states  that  for  an  exponentially  decaying  operator  K the 
rate  of  convergence  depends  primarily  on  the  smoothness  of  the  desired 
solution  and  is  inversely  proportional  to  -Una. 

Theorem  10.  Consider  the  norms  fi  , p^l  and  equation  (1) . Let  the  desired 
P 

solution  f^  have  m>l  derivatives  with  each  f^^^  e 0<j<m,  and  its 

Fourier  transform  ^ satisfy  | | <R^f^r  all  w,  and  D^oj  ' 
for  large  oj. 

Let  K be  of  type  1 with  0<s<l/4  and  v=0.  (v=0  corresponds  to  a finite 
number  of  zeros.)  Moreover,  assume  | ft  (u))  | S D 'ji)  ^ as  u)f“.  Then  for  suffi- 
ciently small  a,  we  obtain  the  following  lower  bound: 

llh^ll^  ^ Qa®  with  a = (2m+l) / (2p+2r)  (65) 

Proof  of  Theorem  10.  As  a+O,  M(a)  in  (60)  becomes  arbitrarily  large. 
Therefore,  for  sufficiently  small  a,  we  have 

||h  ||2  . /°“(aVP^^^|^|2)/4(DtaVP^^>^) 

“ M 

Since  ^=0^0  ^ have  for  io>M 

2 4p+4r  ^ _ 
a 0)  > C„. 


Therefore,  for  some  Q 

M 

with  a“(2m+l)/(2p+2r) . 


Q.E.D. 


Comments . Note  that  for  p<m/2  - r + 1/4,  the  exponent  in  (65)  is  greater 
than  2.  So  we  obtain  an  order  exact  estimate  for  the  rate  of  convergence 


only  for  p>m/2  - r + 1/4,  0ss<l/4  and  v*«0. 
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Next  we  consider  the  condition  of  the  approximating  problems  gener- 
ated. We  again  use  the  dynamic  condition,  Makhoul  LlU. 

Theorem  11.  Consider  the  norms  pSl  and  equation  (1)  . Let  ^ be  of 
type  1 or  type  2.  Assume  moreover  that  |^|  is  a monotone  decreasing  func- 
tion of  u such  that  Dw  ^<|ic(w)|<  Doj  outside  an  interval  of  0 for  type  1, 
and  Cexp(-au)<  |<^(w)  I < Cexp(-aw)  for  all  o)  for  type  2.  Then  the  dynamic 
condition  of  the  members  of  the  approximating  families  generated  is 
of  order 

^^-r/(p+r) 

for  type  1 operators  and  is  of  order 

Qa  ^ ( (-S.nba) /2a) 

for  type  2 operators  where  b=p/ac^ . 

Proof ; We  consider  the  operator  in  the  Euler  equation  (8) , 

4>  = K*K  + aB 


(66) 


(67) 


Part  a.  type  1 operators 

We  have  that  outside  an  interval  of  id*0,  (0,6), 


Du)  s|$(u)  |s  Du) 


The  minimum  of  do)  occurs  when 


0)  = (rd  /2pa) 


l/(2p+2r) 


and  the  corresponding  minimum  value  is 


(68) 


(69) 


For  small  a,  w in  (68)  is  not  in  (0,6).  Therefore,  the  minimum  value 
of  I $ (w) I lies  between  the  2 values  of  (69)  obtained  for  d » D or  D.  As  we 
argued  in  section  3,  |$|  has  no  maximum;  however  in  practice  the  maximum 
is  limited  by  the  sample  size.  Therefore,  we  obtain  (66) . 


31 


Part  b.  type  2 operators 

Using  a similar  argument  for  case  (b)  we  obtain  the  equation 

apu)^^  ^ = ac  exp(-2aw) 

that  we  must  solve.  The  solution  is  approximately 

ui*  = (-inba)/2a  - (Hn  ( (-Unba) /2a)  ) (2p-l) /2a 
where  b=p/ac  and  the  minimum  value  is 

|$(o)*)|  = Qa  ( (-Unbo) /2a) 


Q.E.D. 


Comments.  We  see  in  Theorem  11  that  for  operators  that  decay  like 

w , r>0,  increasing  the  order  of  Q has  a significant  effect  upon  the 

condition  of  the  approximating  problem  generated  by  the  regularization. 

However,  if  the  decay  of  K is  exponential,  then  this  effect  is  not  great. 

As  indicated  in  section  2 for  operators  that  decay  exponentially,  to  achieve 

a condition  similar  to  (66)  we  must  use  a norm  like  0 in  (12)  or  (1  in 

00  00 

(23)  . 

C 

Theorem  12.  Consider  the  norms  (2  and  Q introduced  in  section  2 equa- 
— » - ■■  00  00 

tions  (12)  and  (23).  Let  be  of  type  1 or  type  2,  satisfying  the  hypo- 
theses of  Theorem  11.  Then  the  dynamic  condition  of  the  members  of  the 
approximating  families  generated  is  for  K of  type  1,  of  the  order 

Q((-Hnba)/c)'^  (70) 

C 

where  q=2r  corresponds  to  and  q=r  to 

Q 

For  type  2 operators  we  obtain  for  f)^,  the  order 

Qa  with  d = 2a/(2a+c)  (71) 

and  for  fJ_  the  order 

1/2 


Q(  (-ilnba)/c) 


(72) 


Proof  Part  a.  type  1 operators. 

As  in  the  proof  of  Theorem  11,  we  compute  the  minimum  of 


for  fl  and  of 


= d(i)  + ocoshcw 


d(D  + aexp(caj^) 


(73) 


(74) 


for  fi  . Differentiating  (73)  we  obtain 
00 

2r+l  _ , , • V.  , “1 

acw  = 2rd  (sinhcio) 

But  this  equation  has  the  same  form  as  that  obtained  for  case  (b)  in 
Theorem  11.  Therefore,  its  approximate  solution  is  given  by  (63)  with 


some  renaming  of  constants  and  replacing  a by  a.  Therefore, 

I A , .1 . , 2r 


For  (74)  we  obtain 


|d!^(ii))|  i (c/(-inba))‘ 


2r+2  ^ , 2, 

acw  = rd  exp(-cw  ) 


(75) 


(76) 


The  solution  of  (76)  is  approximately 


cw  = -fn(ba)  - (r+1)  In  ( (-Kn(ba) )/c) 


where  b*c/rd  . Therefore, 


|$tw)|  ^ (c/(-enba))^ 
2 


Part  b.  type  2 operators. 


= d exp(-2aw)  + o exp(cw) 


for  n ; and  for  f)  , 


“ d exp(-2aw)  + a exp(cw^) 
Differentiating  (77)  and  solving  for  w we  obtain 

w A,  - in(l3a)/(2a+c) 


(77) 


(78) 


where  b**c/2aCj^.  Therefore, 


|$j^|  > with  d ■ 2a/(2a+c)  . 
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Differentiating  (78)  and  solving  for  w we  obtain 


Therefore , 


2 1/2 
cw  -Kn(ba)  - Jn  ( (-£nba) /c) 


1/2 

^ (c/(-5.nba))  ^ (1/b) 


Combining  these  estimates  we  can  obtain  overall  estimates  on  the  error, 

r I I 2 


f (a,e) 


O' 


We  conclude  this  section  with  the  following  theorem  which  states  that 

a regularizing  family  with  improved  convergence  and  condition  can  be  used 

instead  of  equation  (8)  if  K is  symmetric  and  positive  definite. 

Theorem  13.  If  the  operator  K in  (1)  is  symmetric  and  positive  definite, 

then  for  p^l  the  solutions  of  the  problems  0<aSl 

(K  + aB)f  = g (79) 

with  the  associated  natural  boundary  conditions  form  a regularizing  family 

for  Kf=g  with  respect  to  0 . 

P 

We  note  that  the  decay  rate  for  K is  the  square  root  of  that  for  K*K. 

Proof  of  Theorem  13.  Define  operators  K =yo^'^^u  u"^  and  K„=  Tu  u'^/o^'^^. 
l^n  nn  2^nnn 

Then  (79)  is  the  Euler  equation  for  the  problem 


Minimize  ||K^f  - K^gM  + afi(f)  . 


Equivalently, 


Minimize  | |Kf  - g| | + afl(f) 

*^3 


(80) 


(81) 


r T 

where  = >u  u /o 

< ^ n n n 


3 n n'  n 

We  first  must  show  that  each  of  the  families  (80)  and  (81)  is  well- 
defined.  First  note  that  the  {u  } are  complete  since  K is  invertible. 
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Moreover,  we  know  that  Kf=g  has  a solution  only  if 

V , T ,2,2 

Therefore,  K^f,  K^g  and  K^g  are  all  well-defined. 

» 

We  know  that  f minimizes  (80)  if  and  only  if  it  satisfies  the  asso- 
a 

ciated  Euler  equation.  But,  due  to  the  symmetry  this  equation  is  just  (79) 
But,  for  example  from  Tikhonov  [13),  we  know  that  the  solution  of  (79)  con 
verges  at  least  uniformly  to  the  solution  of 

Kj^f  = K^g  (82) 

as  a + O.  But  for  the  given  g any  solution  of  (82)  is  a solution  of  (1)  . 

Q.E.D 

Comments . Therefore  for  symmetric,  positive  definite  problems  the  condi- 
tion of  our  approximating  problems,  if  we  use  (79),  is  of  the  order 
with  b=-r/(2p+r),  as  is  the  error  due  to  the  noise  in  the  data. 
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5.  Summary 

In  the  preceding  sections  we  have  considered  the  question,  how  does 
our  choice  of  in  (5)  affect  the  condition  and  the  rate  of  convergence  of 
the  approximating  problems  generated.  For  convolution  equations,  the  re- 
sults obtained  clearly  illustrate  the  relationship  of  the  condition  and 
the  rate  of  convergence  to  the  choice  of  norm  51,  the  character  of  the  com- 
pactness of  the  operator  K,  and  the  smoothness  of  the  desired  solution. 

The  condition  of  the  approximating  problems  generated  is  strongly  depen- 
dent upon  the  order  of  51,  and  improves  as  we  increase  this  order.  For  a 
given  51,  the  condition  deteriorates  as  we  increase  the  decay  rate  of  the 
singular  values  of  K.  The  approximations  are  well-conditioned,  if  we  choose 
an  51  whose  growth  rate  is  at  least  as  fast  as  the  reciprocal  of  the  decay 
rate  of  our  operator.  The  rate  of  convergence  depends  not  only  on  the 
compactness  of  K and  the  order  of  51  but  also  upon  the  smoothness  of  the 
desired  solution.  For  a given  operator  K,  the  smoother  the  desired  solu- 
tion the  better  the  rate  of  convergence.  This  rate  deteriorates  as  either 
the  order  of  51  is  increased  or  the  decay  rate  of  the  operator  is  increased. 
With  respect  to  the  order  of  51,  this  deterioration  is  limited  by  the  smooth- 
ness of  the  desired  solution. 

The  theorems  obtained  give  heuristics  for  determining  the  appropriate- 
ness of  a proposed  regularization  for  a convolution  equation.  In  practice 
we  might  not  know  the  smoothness  of  our  solution  and  would  choose  a lower 
order  51  because  it  is  easier  to  work  with.  Moreover,  we  would  probably  be 
working  on  a finite  interval,  for  example  [0,11  not  (-®,“) . 


36 


In  an  implementation,  we  have  at  least  3 ways  of  soiviny  the  minimi- 
zation problems  in  (5)  : 

(a)  Direct  discretization  of  the  functionals  in  (5)  or  of 
the  Euler  equation  in  (8) . 

(b)  Solving  the  Euler  equation  using  Fourier  transforms. 

(c)  Inverting  B in  (8)  with  the  natural  boundary  conditions 
and  solving  the  resulting  integral  equation  of  the  second  kind. 

All  3 approaches  have  been  used  in  practice,  see  Glasko,  Kulik  and 
Tikhonov  [13]  for  approach  (a),  Anderssen  and  Bloomfield  fl4]  for  approach 
(b)  and  Cullum  [15]  for  approach  (c) . Approach  (a)  is  limited  to  lower 
order  whereas  approach  (b)  could  be  used  with  general  regularizers  as  is 
done  in  Aref’eva  [6],  In  both  approaches  (a)  and  (b) , if  we  work  with  the 
Euler  equation  we  have  the  question  of  satisfying  the  natural  boundary  con- 
ditions associated  with  the  Euler  equation.  The  statement  that  (8)  is  equiv- 
alent to  (5)  includes  the  boundary  conditions.  With  approach  (c)  we  note 
that  the  solution  obtained  automatically  satisfies  the  natural  boundary 
conditions  because  we  must  have 

af  = B"^(K*g  - K*Kf) . (82) 

We  also  note,  however,  that  if  we  use  approach  (c) , then  the  resulting 
operator  in  (82)  is  only  perturbed  by  al  from  the  rapidly  decaying  operator 
B ^K*K.  Thus,  this  approach  does  not  seem  desirable.  However,  we  can 
argue  that  equation  (82)  is  not  as  bad  as  it  seems.  In  contrast  to  what  is 
often  done  in  practice — namely,  smoothing  the  given  data  and  then  solving 
*^^*^smooth””^^  approach  (c)  we  simultaneously  smooth  the  data  B ^K*g  and 
alter  the  equation.  Smoothing  the  data  in  this  way  removes  the  high 
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h- 


frequency  components  of  the  noise  in  a way  consistent  with  the  original 

equation.  To  see  this  consider  the  following.  For  B ^ generated  by  (6) 

the  eigenvectors  have  the  oscillation  property.  That  is,  if  we  order  the 

eigenvalues  of  B y such  that  y 4-0  as  n+®,  then  the  number  of  zero 

n n 

crossings  of  the  eigenvectors  w = sin  mrt  n=l,2...  (for  n=0,  w.=l)  increases 

n 0 

as  n*“.  Since  the  error  in  the  mollified  data  satisfies 

-1  ■'  T 

B K*e  = ) (w  K*e ) p w , 

■-  n n n 
n 

the  high  frequency  components  of  K*e  (and  therefore  of  e)  are  mollified  by 
U . 

We  could  obtain  a symmetrized  version  of  (82) 

((Bj^K*KB^)  + al)  h = B^K*g  (83) 

-1/2 

if  we  could  compute  Bj^=B  . Then  h will  be  derivatives  of  f,  so  the 
solution  of  (83)  may  not  be  as  well  behaved  as  that  of  (82)  . Once  we  have 

1 

(83)  we  can  relate  it  directly  to  the  results  achieved  with  B=identity, 
see  Hilgers  [16].  We  note  that  if  instead  of  using  B with  the  natural 
boundary  conditions,  we  use  another  operator  with  different  boundary 
conditions,  see  for  example  Hilgers  [16],  then  the  approximations  we  gen- 
erate will  be  good  only  if  the  desired  solution  satisfies  these  conditions. 

A comparison  of  the  3 approaches  (a)  - (c)  is  needed. 

The  preceding  discussion  demonstrates  the  importance  of  one's  having 
some  understanding  of  the  given  operator  K.  This  can  be  used  to  estimate 
the  appropriateness  of  any  proposed  regularization,  and  to  give  indication 


of  potential  difficulties. 
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